Third-party watermarking

ABSTRACT

A “third-party watermark” is inserted into a file or files uploaded by a client to a “storing party” such as a file backup server. The third-party watermark may contain information about the upload itself, such as time and date of the upload and the identity of the client. The third-party watermark may also contain authentication information received from the client or elsewhere that establishes that the client is in proper possession of the file, e.g., it is not a “bootlegged” copy.

BACKGROUND

The present invention relates to digital watermarking, also calledcomputational watermarking.

Digital watermarking (hereinafter “watermarking”) is a technique whereininformation is embedded into a digital file in a way which does notnoticeably affect the file contents. Thus a watermark can be insertedinto, for example, a media file such as an MPG audio or video file or aJPG still image file without there being any noticeable change in howthe audio sounds or what the still image or video looks like.

Watermarks can be applied at the time a file is created—for example,when a CD master is made or when a copy of the file is downloaded to apurchaser from an on-line store—and can used for identification andenforcement purposes. The exact content of the information embedded inthe watermark depends on the particular application, but may include theidentity of the creator or the distributor of the work, the identity ofa person to whom the work was sold, a customer account number, or thelicensing terms between seller and purchaser. The information thusembedded can be used to demonstrate content ownership, contentmisappropriation or as a proof of purchase.

Specifically, a rights holder holds intellectual property rights, suchas a copyright, in all or a portion of the contents of many files, suchas music files, video files and so forth. The information embedded inthe watermark can be “authentication information” that establishes thatsomeone in possession of a copy of the file obtained it in a mannerauthorized by the rights holder. Such authentication information couldhave, for example, been inserted into the file at the time of a downloadby an on-line music store or other distributor who was authorized by therights holder to distribute copies of the file and might includeinformation about the transaction, as noted above.

SUMMARY

The present invention is directed to yet another use of watermarks,referred to herein as “third-party watermarking.”

In accordance with the invention, one or more watermarks are insertedinto a body of digital information—such as a text file or an audio orimage file—by an entity or party (the “storing party”) that receives andstores uploaded files from clients for later retrieval by the uploadingparty or by others. This “third-party watermark” would typically beinserted into the file upon receipt by the storing party before it isstored. We use the term “third-party” watermark as a convenient way ofdistinguishing it from a watermark that had already been inserted intothe file—such as by an on-line music store—before it was transmitted tothe client. The uploaded digital information is referred to herein as afile or files, but the term “file” is intended to encompass any body ofdigital information that a client may upload to the storing party andthat is capable of being digitally watermarked, whether or not that bodyof digital information meets any strict definition of the term “file.”The file and/or information represented by the digital data of the fileis also sometimes referred to herein as “content.”

The storing party may be, for example, a file backup service to whichclients upload files over the internet. If a client's copy of the fileis lost due to computer malfunction or theft, the client is able todownload a replacement copy of the file from the file backup service.Another storing party that could advantageously use the invention is aweb hosting company that receives and stores uploaded web page files,e.g., html files.

The information embedded in the third-party watermark by the storingparty can be of one or more different categories of information. Theinformation in the various categories might all be embedded in onethird-party watermark, each in its own respective different third-partywatermark, or some combination of these.

One such category may be information relating to the upload itself, suchas information identifying the uploading client and the time and date ofthe upload.

Another such category may be information—herein referred to as“authentication data”—establishing that the client is rightfully inpossession of the file. The authentication data could be in variousforms, such as: a JPG image of a paper receipt or an emailed receiptreceived by the client from an on-line music store or other distributorauthorized by the rights holder. While better than nothing, suchauthentication data may be subject to question by an inquiring partybecause such authentication data would be easy to forge. However, inaccordance with a feature of the invention, the storing party may obtainproof-of-purchase or other authentication data directly from theclient's source (e.g., the on-line music store), the payment agent(e.g., credit card company that issued the credit card that was used topurchase the download from the on-line music store), or some otherindependent party. The authentication data may be not aproof-of-purchase per se because the client may have subscribed to anunlimited-download account with the on-line music store wherein theclient is entitled to download as much music as s/he desires for a flatfee. The authentication data will in any event, however, be such as toestablish that the client's copy was obtained from an authorized source.

The insertion of a third-party watermark into the file prior to itsbeing stored by the storing party can benefit both the storing partyitself and the uploading party.

As one example as to how the storing party can benefit, consider thepossibility that a hacker hacks into the storing party's server in whichthe uploaded client files have been stored; retrieves stored mediafiles, such as audio recordings or movies; and then redistributes them.If these “bootleg” copies are somehow traced back to the storing party,the storing party will be able to establish that it had not itselfillegally obtained the copies but, rather, had accepted them in goodfaith on such-and-such a date from such-and-such a client. If thethird-party watermark also contains authentication data, so much thebetter in terms of establishing that the file was obtained by the clientin an authorized way.

As one example as to how the client can benefit from third-partywatermarking, consider the fact that record industry spyware or othersoftware may peruse the contents of the client's computer looking forunauthorized copies of copyrighted content, such as bootlegged musicfiles. The client would like to have ready proof, perhaps even directlyreadable by the spyware, that music stored in the client's computer wasproperly obtained.

So now assume that the client had legitimately obtained a copy of amusic album from an on-line store; then uploaded to the storage servereither the entire album or individual tracks “ripped” from the album;and thereafter downloaded the album or tracks back down into theclient's computer. And now consider the possibility that the client isaccused of being in possession of a bootlegged copy.

In certain circumstances, on the one hand, the client may not requirethe benefit of third-party watermarking. That is, the client may be ableto readily establish on his/her own that the client's copy of the albumor tracks is legitimate. For example, the client may have a physical oremailed receipt; or the client may be able to locate in her personalrecords a transaction number that can be used to recover informationabout the transaction from the on-line store's database or from a creditcard company or other payment agent whose services were used toeffectuate the purchase transaction; or a watermark with embeddedauthentication information might have been inserted into the album orthe individual tracks by the on-line store at the time of download.

In other circumstances, however, none of these forms of proof may beavailable. The client may have lost the receipt; the original downloadedfiles may never have been watermarked by the on-line store; or theon-line store may have gone out of business, making it no longerpossible to look up the transaction.

Advantageously, however, if the client had, say, provided thetransaction number to the storing party at the time of the upload (whichis assumedly before the client lost track of the transaction number),the storing party could have at that time obtained relevantproof-of-purchase or other authentication information from the on-linestore (assuming that the store was set up to receive such requests) andthen the storing party could have inserted the authenticationinformation into the third-party watermark, where it would always beavailable for inspection.

In particular scenarios in which the invention might be used, thestoring party—call it the XYZ File Backup Service—may enter into anagreement with one or more on-line stores whereby, when a media file orother file is obtained on-line, the on-line store asks the client if theclient would like to have the file backed up by XYZ File Backup Service.If the client elects to have this done, a copy of the file could be sentnot only to the client but to the XYZ File Backup Service, along withauthentication information that the file backup service could then embedin the third-party watermark.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an illustrative system in which the presentinvention may be implemented;

FIG. 2 is a schematic representation of a media file into whichwatermarks have been inserted; and

FIG. 3 is a flow chart of a method illustratively implemented in thesystem of FIG. 1 in carrying out the invention.

DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT

FIG. 1 shows a client's computer 11 connected to the internet. Computer11 is representative of a multiplicity of internet-connected computers,each of which may individually upload files to a file backup server asdescribed herein.

The client may create a new audio, video, text or other file locally oncomputer 11 or may obtain it from any of various sources. One suchsource is on-line music store 12, from which the client may downloadmusic files pursuant to some payment arrangement with the on-line musicstore, such as pay-per-download or unlimited download. Also connected tothe internet is a payment agent 13, which might be a credit card companyor the client's bank. Purchases from the on-line store may involve theservices of payment agent 13 to pay for the downloaded music.

The remaining element shown in FIG. 1 is file backup server 15. Thisserver is operated by a file backup service which enables clients toupload media files, as well as any other desired type of file, to filebackup server 15, where they are stored in storage memory 155. Clientscan later download selected one(s) of their files from server 15, e.g.,in the case where files were inadvertently erased or otherwise lost.File backup server 15 further includes server hardware 151, whichincludes one or more processors and associated peripherals and othercomponents typically included in such a server. File backup server 15further includes backup application 153. This is software which, whenexecuted by the server hardware, implements the file backup service asjust described. File backup server 15 further includes watermarkingapplication 152. This is software which, when executed by the serverhardware, creates and inserts watermarks into files that are uploaded tothe server by clients.

File backup server 15 may be any appropriate server hardware purchasedfrom a commercial vendor. One typical such server is the SPARCEnterprise T5140 manufactured by Sun Microsystems, Inc.

FIG. 2 is a conceptual representation of a media file, taken asillustrative. The media file includes media content 21, which is MPEG-,JPG- other encoded or non-encoded bits representing audio and/or imageinformation. Two watermarks have been inserted into media content 21.One of these is a download-transaction watermark 22 that may have beeninserted by a file source, such as an on-line store, prior todownloading the file to a purchaser or other client. Thedownload-transaction watermark may include various pieces of informationabout the transaction, such as information identifying the source, theclient, the time and date of the transaction, etc. Certain filesuploaded to the storing party may include such a download-transactionwatermark, but some may not.

The other watermark inserted into media content 21 is third-partywatermark 24, as described herein. This third-party watermark wasinserted into the file content—illustratively watermarking application152—at the time that the file was uploaded to server 15.

Watermarks 22 and 24 are depicted as discrete portions of the mediacontent 21 for pedagogic simplicity. As is well known, however, suchdigital watermarks are created through a computational, algorithmicprocess that modifies the bits that represent the media content. This isdone in such a way that the modification of the media content does notaffect to any significant extent how the media sounds or looks to thenormal listener but yet allows the information embedded into thewatermark to be retrieved from the file. The particular watermarkingalgorithm used in server 15 is of no consequence to the presentinvention. Any known or future-developed algorithm can be used. Typicalof current state-of-the-art watermarking algorithms are those describedin Digital Watermarking and Steganography, 2^(nd) ed., by Ingemar J.Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich and Ton Kalker(pub. Morgan Kaufmann, 2007; ISBN 0123725852).

FIG. 3 is a flowchart of a method illustratively implemented in thesystem of FIG. 1 in carrying out the invention. The various componentsof the method are performed by the client's computer 11, the on-linestore 12 and the file backup server 15, as FIG. 3 denotes.

Initially, the client creates a file or files, as indicated at 301, ormay obtain a file from elsewhere, as indicated at 303—in this example bymaking a purchase at an on-line store. The client may, for example, usea credit card to pay for an individual download, or the client may havean unlimited-download or package subscription with the on-line store, inwhich case the obtaining of the music file is not a purchase per se. Inany event, the on-line store, in due course, downloads the file to theclient, as indicated at 306.

It is assumed that the client has already established an account withthe file backup service. As part of the interaction with the on-linestore, the client may have elected to have the just-obtained file backedup at file backup server 15, as determined at 308. In that case, asindicated at 315, the on-line store sends a copy of the file to filebackup server 15, along with information identifying the client andinformation about this transaction, such as a transaction number orother information, all which is received by the file backup server asindicated at 322. File backup server 15's further processing of the datajust received is described at a more opportune point below.

We return at this point to the client. As indicated at 311, the clienthas decided to back up a certain file in computer 11 by uploading thatfile to server 15. As part of the interaction with server 15, the clientis prompted at 317 to indicate whether the client has anyproof-of-purchase or other authentication information relative to thefiles being uploaded.

That authentication information is illustratively one of two types inthis example.

In particular, the client may have an image file showing an image of areceipt or other proof that the client has paid for, or is otherwise inproper possession of, the file to be uploaded. Such proof would beevidence that the file is not a bootleg copy. In that case, the clientmight like to upload that image file along with the content itself sothat it can be embedded in the third-party watermark that will beinserted by the file backup service. Alternatively, as indicated at 317,the client may have information that points to proof-of-purchase orother authentication data, such as a transaction record number that wasissued by the on-line store, or by the payment agent, or perhaps by aservice that functions as a central clearinghouse for such transactionrecords. If any of these is the case, the client may supply thetransaction number to file backup server 15. The server, inturn—pursuant to a prearranged relationship with the on-line store,payment agent or transaction record service—obtains theproof-of-purchase or other authentication information from theappropriate source, as indicated at 325.

The client may not have any proof-of-purchase or other authenticationinformation. Receipts and/or transaction numbers may have been lost; thecontent may, indeed, be a bootlegged copy; or the content may have beencreated by the client or might have been received from elsewhere withoutany restriction on further dissemination. For example, the file may be asimple text file or a homemade media file created by the client'sbrother's after-school rock band. In all such cases, obviously, theclient will upload the file without any proof-of-purchase or otherauthentication information.

We have to this point seen three different scenarios in which a file,with or without authentication credentials, has arrived at server 15—a)content created by the client or received by the client withoutrestrictions on further dissemination; b) content obtained by the clientthat IS subject to restrictions, such as a copyrighted music file thatthe client received from an on-line store; and c) a file that, pursuantto client wishes, the server received directly from the on-line store,along with proof-of-purchase or other authentication information.

These three scenarios coalesce at 329, where server 15 generates uploadtransaction information pertinent to the upload transaction itself. Thisupload transaction information may thus include such pieces of data asthe time and date and the source of the file, such as a) the identity ofthe client (or at least an account number that the file backup servicehas established for the client), or b) the identity of the on-line storethat had sent the file at 315.

Watermarking application 152 thereupon inserts one or more third-partywatermarks into the file, as indicated at 332. These one or morethird-party watermarks embed into the file whatever upload transactioninformation and/or b) authentication information has been generatedand/or received by file backup server 15. The resulting watermarked fileis then stored at 334 in backup storage 155 from which it can later bedownload to the client on request, as indicated at 336.

The foregoing merely illustrates the invention and many variations arepossible. For example, although the discussion herein often makesreference to media files, any type of file that, now or in the future,is able to be digitally watermarked can have inserted therein athird-party watermark pursuant to the principles of the presentinvention. The use of the term “client” is intended to be generic to anyperson who uses or owns a computer or other instrumentality that iscapable of uploading files to a storing party as described here.

Although the various components of the system are shown in FIG. 1 asbeing in communication with one another over the internet, any otherform of network or electronic communication medium or infrastructure maybe involved in the various communications, particularly those betweenthe client and the file backup server.

The term “server” is intended to encompass any apparatus, system orcombination of these—be it hardware, software (in any of its variousforms such as a firmware) or any combination of these—that is capable ofcarrying out the functions described herein as being performed by filebackup server 15.

Although in the illustrative embodiment the storing party is a filebackup service operating a file backup server, the invention is equallyusable by any entity that receives files and wants to embed into thosefiles—for its own benefit or for the benefit of the uploading party—oneor more third-party watermarks as described herein. Web hostingcompanies and internet service providers are examples of entities thatreceive and store such files.

In the illustrative embodiment, the third-party watermark is insertedinto the uploaded file at the time of upload. However, the inventionencompasses the possibility that the storing party may desire to insertthe watermark at some time after the file was uploaded and initiallystored and/or may desire to insert one or more new or modifiedthird-party watermarks to, for example, embed new information into thefile, such as authentication information that the client did not provideat the time of the original upload.

Some of the claims hereof recite certain elements in means-plus-functionterms. At the heart of applicants' invention is the notion of insertingthird-party watermarks into files as described herein. Thus what is atthe heart of the invention is a new combination of functionalitiesbrought together to achieve one or advantages in a new and non-obviousway rather than the use of any particular equipment or apparatus toimplement those functionalities. Thus applicants regard any equipment orapparatus that is capable of carrying out the functions called for inthe various means-plus-function recitations of the claims as beingequivalent to those which are explicitly shown and/or described herein.

It should thus be understood that those skilled in the art will be ableto devise various alternative arrangements and steps that, although notexplicitly or implicitly shown or described herein, implement theprinciples of the invention and are therefore within the spirit andscope of the invention as claimed.

The invention claimed is:
 1. A method performed by a file backup serviceimplemented on a server that receives files from at least first, secondand third clients over the internet and stores the received any type offiles for the clients in such a way that the clients are able todownload replacement copies from the file backup service, the methodcomprising the file backup service receiving from the first clientduring at least one upload transaction an uploaded first digital filecapable of being digitally watermarked, the uploaded first digital filecontaining content for which a first rights holder holds copyrightrights in all or a portion of the contents of the first digital file,the file backup service prompting the first client for authenticationinformation that can be used to establish that the first digital filewas rightly obtained by the first client, the file backup servicereceiving from the first client a transaction identifier identifying atransaction in which the first client had purchased the first digitalfile from an on-line store that sells authorized copies of digital filesto purchasers, the file backup service transmitting the transactionidentifier received from the first client to the on-line store, the filebackup service receiving from the on-line store, in response to thetransmission of the transaction identifier, proof-of-purchaseinformation establishing that the first client had purchased the firstdigital file from the on-line store, the file backup service digitallywatermarking the first digital file with a third-party watermark thatcontains the proof-of-purchase information establishing that the firstclient had purchased the first digital file from the on-line store, thefile backup service downloading the third-party-watermarked firstdigital file to the first client, the file backup service storing thethird-party-watermarked first digital file, the file backup servicereceiving from the second client during at least a second uploadtransaction an uploaded second digital file capable of being digitallywatermarked, the uploaded second digital file containing content forwhich a second rights holder holds copyright rights in all or a portionof the contents of the second digital file, the file backup serviceprompting the second client for authentication information that can beused to establish that the second digital file was rightly obtained bythe second client, the file backup service receiving from the secondclient an image of a sales receipt establishing that the second clienthad purchased the second digital file from said on-line store, the filebackup service digitally watermarking the second digital file with athird-party watermark that contains the image of the sales receiptestablishing that the second client had purchased the second digitalfile from said on-line store, the file backup service downloading thethird-party-watermarked second digital file to the second client, thefile backup service storing the third-party-watermarked second digitalfile, the file backup service receiving from said on-line store a thirddigital file capable of being digitally watermarked, the third digitalfile containing content for which a third rights holder holds copyrightrights in all or a portion of the contents of the third digital file,the third digital file having been transmitted from said on-line storeto the file backup service in response to an indication from the thirdclient, who had obtained the third digital file from the on-line store,that said third client wanted to have the third digital file backed upby the file backup service, the file backup service further receivingfrom said on-line store proof-of-purchase information establishing thatthe third digital file was purchased by the third client from theon-line store, the file backup service digitally watermarking the thirddigital file with a third-party watermark that contains theproof-of-purchase information establishing that the third digital filewas purchased by the third client from the on-line store, and the filebackup service storing the third-party-watermarked third digital file.2. The method of claim 1 further comprising in response to a hacker a)hacking into a server of the file backup service and b) redistributingthe third-party-watermarked first, second and third digital files, thefile backup service utilizing the third-party watermarks in thethird-party-watermarked first, second and third digital files to provethat the first, second and third digital files uploaded to the filebackup service by the first, second and third clients, respectively,were properly obtained by the first, second and third clients.
 3. Themethod of claim 1 further comprising, in response to discovery of thethird-party-watermarked first, second and third digital files by spywarelooking for unauthorized copies of copyrighted content in computers ofthe first, second and third clients, using the third-party watermarkstherein to establish that the first, second and third digital files hadbeen properly obtained by the first, second and third clients,respectively.
 4. A file backup server of a type that receives files fromat least first, second and third clients over the internet and storesthe received any type of files for the clients in such a way that theclients are able to download replacement copies from the file backupservice, the file backup server comprising file backup server hardwarethat includes at least one processor, and a non-transitory storagemedium storing program instructions that, when executed by said at leastone processor, cause the file backup server to carry out the steps ofreceiving from the first client during at least one upload transactionan uploaded first digital file capable of being digitally watermarked,the uploaded first digital file containing content for which a firstrights holder holds copyright rights in all or a portion of the contentsof the first digital file, prompting the first client for authenticationinformation that can be used to establish that the first digital filewas rightly obtained by the first client, receiving from the firstclient a transaction identifier identifying a transaction in which thefirst client had purchased the first digital file from an on-line storethat sells authorized copies of digital files to purchasers,transmitting the transaction identifier received from the first clientto the on-line store, receiving from the on-line store, in response tothe transmission of the transaction identifier, proof-of-purchaseinformation establishing that the first client had purchased the firstdigital file from the on-line store, digitally watermarking the firstdigital file with a third-party watermark that contains theproof-of-purchase information establishing that the first client hadpurchased the first digital file from the on-line store, downloading thethird-party-watermarked first digital file to the first client, storingthe third-party-watermarked first digital file, receiving from thesecond client during at least a second upload transaction an uploadedsecond digital file capable of being digitally watermarked, the uploadedsecond digital file containing content for which a second rights holderholds copyright rights in all or a portion of the contents of the seconddigital file, prompting the second client for authentication informationthat can be used to establish that the second digital file was rightlyobtained by the second client, receiving from the second client an imageof a sales receipt establishing that the second client had purchased thesecond digital file from said on-line store, digitally watermarking thesecond digital file with a third-party watermark that contains the imageof the sales receipt establishing that the second client had purchasedthe second digital file from said on-line store, downloading thethird-party-watermarked second digital file to the second client,storing the third-party-watermarked second digital file, receiving fromsaid on-line store a third digital file capable of being digitallywatermarked, the third digital file containing content for which a thirdrights holder holds copyright rights in all or a portion of the contentsof the third digital file, the third digital file having beentransmitted from said on-line store to the file backup service inresponse to an indication from the third client, who had obtained thethird digital file from the on-line store, that said third client wantedto have the third digital file backed up by the file backup service,further receiving from said on-line store proof-of-purchase informationestablishing that the third digital file was purchased by the thirdclient from the on-line store, digitally watermarking the third digitalfile with a third-party watermark that contains the proof of purchaseinformation establishing that the third digital file was purchased bythe third client from the one-line store, and storing thethird-party-watermarked third digital file.